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Abstract 

The efficiency of an experimental design is ultimately measured in terms of time 
and resources needed for the experiment. Optimal sequential (multi-stage) design 
is studied in the situation where each stage involves a fixed cost. The problem is 
motivated by switching measurements on superconducting Josephson junctions. In 
this quantum mechanical experiment, the sequences of current pulses are applied to 
the Josephson junction sample and a binary response indicating the presence or the 
absence of a voltage response is measured. The binary response can be modeled by 
a generalized linear model with the complementary log-log link function. The other 
models considered are the logit model and the probit model. For these three models, 
the approximately optimal sample size for the next stage as a function of the cur- 
rent Fisher information and the stage cost is determined. The cost-efficiency of the 
proposed design is demonstrated in simulations based on real data from switching 
measurements. The results can be directly applied to switching measurements and 
they may lead to substantial savings in the time needed for the experiment. 
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1 Introduction 



In binary response experiments carried out in various fields of experimental 
science, the researcher chooses the value of a covariate variable and measures 
a binary response. The measurement data is modeled by a binary response 
model where the probability of success or failure is assumed to be a monotone 
function of the covariate. Three parametric models, the logit, the probit and 
the complementary log-log (cloglog) model are frequently used to model the 
dependence between the binary response Y and the continuous covariate x. All 
these models can be presented in the framework of generalized linear models 



(IMcCullagh and Neldeii Il989h 



P(Y=l) = E(Y) = F(ax + b), (1) 

where the response curve F is a cumulative distribution function (cdf ) and the 
a and b are the parameters of the model to be estimated. The three response 
curves commonly used are: logistic distribution for the logit model 

„, , . expfax + b) 

F(ax + b) = '—, 2 

v ; l + exp(ax + 6)' V ; 

normal distribution for the probit model 

F(ax + b) = <5>(ax + b), (3) 

where $ is the cdf of the standard normal distribution, and the Gompertz 
distribution for the cloglog model 

F(ax + b) = 1 — exp(— exp(ax + &)). (4) 



The problem of optimal design arises when the covariate values need to be 
chosen in optimal manner to estimate some parameters of interest. In binary 
response models (1), the parameters of interest are a and b that determine the 
slope and the location of the response curve. Several optimality criteria have 
been proposed in the statistics literature. Among the most popular are D- 
optimal designs, which minimize the generalized variance. The minimization 
of the generalized variance equals to the maximization of the determinant 
of the Fisher information. The expected Fisher information of the data Xj, 
i — 1, . . . , n under the binary response model (1) is defined as 
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where z% = axi + b and 
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for logit model (2), 

for probit model (3) and 



for cloglog model (4) 
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Now the D-criterion can be defined as the square root of the determinant of 
Fisher information (5) 

(9) 



D = ^/det(J). 

The (locally) D-optimal design is derived by maximizing D under the as- 
sumptions that n — > oo and the true values of the the parameters a and 
b are known. The derivation i s non-trivial and can be found in the gen- 



eral form i n (IFord et all I1992T). Th e wide ly used lo g it mo d el (2) is stud- 



ied also bv lAbdelbasit and Plackettl (ll983h : iMinkinl (119871 ): ISitter and Wu 
( 119931 ): iMathew and Sinhal (120011 ). The above references show that the D- 
optimal design is a two-point design for the logit, the probit and the cloglog 
models and the optimal covariate values x* can be solved from the equation 
ax* +b = z*, where the values of the canonical parameter z* are reported in Ta- 
ble 1. Extensions to the models with more than one covariate are considered 



bv IWoods et all ((20061) 
fl2008h . 



Dror and Steinberg (120081 ) and iDorta-Guerra et al 



Table 1 

D-optimal covariate values for the models (2), (3) and (4). The optimal covari- 



ate values x* and x\ are solved from equations ax\ + b 



and ax*. + b 



Columns F(z\) and F(z%) report the cdf values related to the optimal covariates, 
i.e. probabilities of response 1. 

model Optimal covariates 
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F{zl) 


H4) 


logit 


-1.543 


1.543 


0.176 


0.824 


probit 


-1.138 


1.138 


0.128 


0.872 


cloglog 


-1.338 


0.980 


0.231 


0.930 



The efficient use of resources is not measured by the D-criterion but in terms of 
time and resources used for the experiment. Optimal sequential (multi-stage) 
design is studied in the situation where there is a fixed cost for each stage. As 
a motivati ng example , switching measurements on superconducting Jos e phson 



junctions (jJosephsonl . Il962l ; iGrabert and Devoret I . lEds.l ; iPekola et all 120051 ) 



are considered. In this quantum mechanical experiment, the experimenter 
varies the height of the applied current pulse and observes the presence or the 
absence of the voltage response generated by macroscopic quantum tunnel- 
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ing (ICaldeira and Leggettl . 119801) . An experimental design for switching mea- 
surements was proposed by iKarvanen et al.l (120071 ). The following properties 
are characteristic for switching measurements: 



The data acquisition is fast and th e total number of obse rvations can be 



high. In the experiment reported in (IKarvanen et al.l . 120071 ). the total num- 



ber of observations was over 100000 and the experiment took about 8 min- 
utes. This means that there is no time for expert analysis during the exper- 
iment but automated processing must be used. 

Sequential/multi-stage designs are applicable. There is, however, a cost re- 
lated to the number of the stages which usually makes the fully sequential 
approach inefficient. 

The measurement system remains stable only for a limited time because 
very low temperatures are needed for superconductivity. This emphasizes 
the need for cost-efficient designs. 

Prior information on the model para meters may be very poor . 



Differently from dose- response trials (IBiedermann et al.l . 120061 ) . ethical con- 



siderations do not restrict the choice of the design. 

Sensitivity to the modeling assumptions and the need of initial estimates are 
the well-known disadvantages of D-optimality. In switching measurements, 
however, there a r e goo d reasons to use D-optimality: First, it can be shown 



( iKarvanen et al.l . 120071 ) that the cloglog model (4) provides a very good ap- 
proximation for the probability of switching as a function of the height of the 
applied current pulse. In other words, in switching measurements, the true 
model is actually known. Second, the experiment is carried out sequentially, 
which means that poor initial estimates will not spoil the whole experiment. In 
sequential switching measurements, a number of current pulses with a certain 
height are generated, responses are measured for each pulse and finally the 
height of the pulse for the next stage is calculated. Because the configuration 
of the pulse generator takes time, it is not practical to change the height of 
the pulse after every pulse. The total cost equals to the time used for the 
experiment and consist of the time used for measuring and computing and 
the time used for the configuration of the pulse generator between the stages. 

The optimal number of measurements per stage in sequential designs with a 
fixed stage cost is determined for the logit, the probit the cloglog models. The 
proposed solution is approximal and based on numerical analysis. It is assumed 
that the initial maximum likelihood estimates for the parameters are available. 
If the initial estimates are n ot availab l e, the y can be found, for instance, by 



the binary search algorithm (jKarvanenl . 120081 ) . After the initial estimation, the 



experiment continues as a sequential design where the D-optimality criterion 
is used. The optimal stage size, i.e., the optimal number of measurements per 
stage, is derived as a function of the stage cost Cs and the observed value 
of D. The derived cost-efficient designs can be presented in simple functional 
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form and are easily applied to practical experiments. 



Most of the existing works on sequen tial d esign do not consider the cost of 
the experiment as a design criterion. IWul (119851 ) considered fully sequential 
designs where the measurements are done one by one. He criticized the ear- 
lier a pproaches: the stochastic appr oximation method (IRobbins and Monro . 



195ll ) and the up-and-down method (IDixon and Moodl . Il948l ) and proposed a 



sequential approach to the estimation of lOOpth percentile. In this approach, 
the next covariate point x n+ i is chose n so that F n (x„+i) = p, where F n is the 
estimated cdf of logistic distribution. Ilvanova et al.l (120031 1 considered several 
designs from the family of up-and-down rules for the sequential allocation of 
dose levels to subjects in a dose-re s ponse study. The use of group up-and- 
down designs (IGezmu and Flournoyi . 120061 ) implies that there is a cost related 
to the number of stages also in d ose-finding experiments a lthough this cost is 
not usually explicitly considered. iMcLeish and Toshl (Il990[ ) studied sequential 
design in bioassay and used a co st function that in c luded unit cost plus extra 
cost f or each positive response. ISitter and Forbes! (119971 ) and ISitter and Wu 
( 119991 ) studied two-stage designs for the logit and the probit model and used 
the second stage to balance the first stage. The second stage design was 
found by numerical optimization. The results can be extended to the multi- 
stage designs . Theoretical results on asympt otics of sequential designs were 
provi ded in (jChaudhuri and Myklandl . Il993l ) and (jChaudhuri and Myklandl . 
1995 ). A review of Bayesian approach to sequential designs was given by 



Chaloner and Verdinellil (ll995l ). A recent major contribution to the sequen- 



tial design was the procedure proposed by iDror and Steinberg! ( 120081 ). Their 
procedure is based on a Bayesian analysis that exploits a discretiz ation of the 



param eter space to efficiently represent the posterior distribution. iTekle et al. 



(200 81) studied maxim in D-optimal designs for binary longitudinal responses. 



Karvanen et al.l (120081 ) considered two-stage designs for gene-disease associa- 



tion studies where the motivation for the design arises from cost considera- 
tions. 



In Section 2, cost-efficient sequential designs are derived assuming that the 
initial maximum likelihood estimates of the parameters exist. In Section 3, 
application to switching measurements is presented. Discussion in Section 4 
concludes the paper. 
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2 Cost-efficient sequential designs 

2.1 Problem definition 

Consider a sequentially performed experiment where the total cost of the ex- 
periment consists of two components: a cost related to the number of measure- 
ments and a cost related to the number of stages. Without loss of generality, 
the marginal cost of making one additional measurement is fixed as unity 
and the cost of having one additional stage is marked by C5. Thus, the total 
cost of an experiment with total of n measurements in K stages would be 
C = n + KCs- The number of measurements at stage k is marked by n k , 
k — 1, . . . , K. It follows that J2 k =i n k — n - Assume that after k — 1 stages the 
maximum likelihood estimates a k _i and b^-i and the related Fisher informa- 
tion matrix Jfc_i are obtained. On the basis of the estimates a k -\ and b k ^\ and 
the results in Table 1, the estimated D-optimal covariate points x kl and x* k2 
are calculated. The problem is to decide the optimal number of measurements 
n k to be made in the points x* kl and x* k2 . For practical reasons, it is assumed 
that n k is an even number so that the measurements can be divided equally 
between x* kl and x* k2 . Because a k -\ and b k -i are only estimates of the true 
parameters, the covariate points x kl and x* k2 may be far from the actual truly 
D-optimal covariate points. If n k is small, a and b are updated quickly and the 
number of measurements needed is small but the number of stages needed is 
high. On the other hand if n k is large, the number of measurements needed is 
higher but the number of stages needed is smaller. Obviously, the optimal n k 
depends on the stage cost Cs and the Fisher information J k -i- 

The scheme described above approximates the cost structure of switching 
measurements. In switching measurements, there are actually three types of 
costs involved: 

marginal unit cost time needed for making one additional measurement 

with the present pulse height, 
configuration cost time needed for changing the pulse height, 
computing cost time needed for updating parameter estimates. 

The marginal unit cost and the configuration cost remain almost constant dur- 
ing the experiment but the computing cost depends on the details of numerical 
optimization and may vary from stage to stage. Additional complications fol- 
low from the fact that the D-optimal design is a two-point design implying 
that the signal generator needs to be configured when moving from one design 
point to another. This does not prevent using the described scheme because 
the stage cost may be defined as a sum of the computing cost plus two times 
the configuration cost. 
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The problem of cost-efficient design may be expressed as an optimization prob- 
lem with respect to a precision target or a budget constraint. If a precision 
target ^target is used, the optimal n k is found as a solution to the following 
minimization problem 

argmin E (C(x, y) | ^(x, y) > ^target, x*-i, Yk-i, n k ) , (10) 

where ^k-i and y^-i present all measurement data available after k — 1 stages, 
C(x, y) is the cost of collecting measurement data (x, y) and ^(x, y) is the 
value of the precision criterion calculated from the measurement data (x, y). 
For instance, D may be used as the precision criterion. The expectations in 
minimization (10) are integrals over all possible measurement data (X&, Y k ) 
available after k stages. For data (K k = x k ,Y k = y k ) the expected cost to 
reach the precision target is given by 



kC S + J2 n i+ min E (C( X >y) I ^(X,y) > ^target, X fc ,y fc ) , (11) 
1=1 



where the last term in the summation is the expected cost to reach the pre- 
cision target on the condition of data (x k ,y k ) when the optimal strategy is 
used for the rest of the experiment. In the other words, minimization (10) 
involves recursive calculations where the same minimization is performed for 
the subsequent stage sizes n k+ i, n k+2 , .... 

Alternatively, if a budget constraint Cbudget is used, the optimal n k is found 
as a solution to the following maximization problem 

argmax E (^(x, y) | C(x, y) < CWdgct, x fe _i, y fc _i, n k ) . (12) 
Maximization (12) requires similar recursive calculations as minimization (10). 



In stochastic optimization (IBirge and Louveauxi . 119971 ). the problems (10) and 



(12) may be characterized as multi-stage recourse problems where the number 
of the stages is not fixed. Because of the nonlinearity of the objective functions 
and the recursive structure of the problem, it seems difficult to find optimal 
n k in analytical means. The high number of branches in the recursion implies 
that straightforward simulations cannot solve the problem either, at least not 
in reasonable time. To overcome these difficulties, our solution uses numerical 
computation and simplifying approximations. 



2.2 Design paths in the (D,C) -plane 



Because it is difficult to derive cost-efficient sequential designs analytically, 
approximations and simulations are used to obtain designs that are approxi- 
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mately cost-efficient. The first approximation made is to compress the infor- 
mation matrix J into the square root of its determinant D. This allows to 
compare candidate designs in the (D, C)-plane. Each design is characterized 
by the number of measurements n\,ri2, ■ ■ ■ ,njc per stage and can be drawn 
as a path in the (D, C)-plane so that each point of the path represents the 
expected value of D as a function of C. At the beginning of each stage there 
is a vertical jump in the path corresponding to the stage cost C$- After that 
D is expected to increase with a constant slope as C increases. If two addi- 
tional measurements are made, the expected change in D is denoted h(D ), 
where D is the value of D at the beginning of the stage. The number of the 
additional measurements is two because the D-optimal designs are two-point 
designs. An approximation for the function H(Dq) is obtained by means of sim- 
ulation. The idea is to simulate the distribution of a and b on the condition 
that the expected square root of the determinant of the information matrix 
is D Q . Without loss generality it is fixed a = 1 and 6 = 0. The information 
matrix is taken to be 

Jo = ^J*, (13) 



where 



D* 

/, 



(14) 



g{zt){zlf + g(z* 2 )(z* 2 f g(z{)zt + g(z* 2 ); 

\ 9(4)4 + 9(4)4 9(4) + 9(4) 

which is the information matrix of the D-optimal covariates, and D* = \J det J* 
0.8094. An approximation that a and b are normally distributed with the mean 
vector (a, b) and the covariance matrix Jq 1 is made. In the other words, it is 
assumed that the estimates a and b originate from previous D-optimal mea- 
surements. The value of det (J (a, b)) is calculated for each generated pair of a 
and b 



det(J(a,6)) = ^g(z l )g(z 2 )(z l -z 2 )\ (15) 

where 

4~ h A 4~b , * 

z\ = — - — and Z2 = — - — . (lb) 
a a 

The expected change h(D ) is then obtained as avera ge over the values of 



det (J (a, b)). All numerical calculations are done using R (IR Development Core Team! . 



20061 ) . The obtained function h(Do) is plotted in logarithmic scale in Figure 1 
for the cloglog model. When D increases, h(D ) approaches its theoretical 
maximum 0.8094 that corresponds to making the two additional measure- 
ments at the D-optimal points z\ and z\. The graphs for the logit and the 
probit model are similar (not shown). 

In order to use the expected change of the D-criterion in the optimization, 
the function h(D ) need to be interpolated. By looking Figure 1 it is found 
that h(D ) can be roughly approximated by logistic function .D*/(l + exp(77 + 
#log(-Do)))> where r] and 9 are parameters estimated from the simulated data. 
Using this approximation as the basis of the model, a generalized additive 
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Fig. 1. The expected information increase as a function of current information for the 
cloglog model when two additional measurements are made. The solid line represents 
the average of h(Do) in 400000 simulation runs and the dashed lines represents 
1st, 5th, 10th, 20th, 50th, 80th, 90th, 95th and 99th percentiles. The theoretical 
maximum of h(Do) corresponding to the D-optimal two-point design is marked by 
the dotted line. 

model ( Hastie and Tibshirani . 1990l : Hastie . 2006 ) is fitted to the simulated 
data. In the model, smoothing splines with the degree of freedom 8 are used 
to model the nonlinearities remaining after the logistic approximation. The 
estimated model fits very well to the simulated data so that in Figure 1 it 
would not be easy to separate the observed h(D ) from the modeled h(D ) 
(the maximum absolute difference between the observed and modeled values 
of h(D ) is 0.016). 



2.3 Comparing design paths 



Next the optimal number of measurements for the next stage are calculated 
providing that the current point in the (-D, (7) -plane is (D^-x, Ck-i) and the 
stage cost is Cs- For the calculation a two-stage approximation that evaluates 
a large number of candidate paths after two stages is used. The candidate 
paths are chosen such a way that the potential values of are covered. It is 
assumed that our precision target (defined in terms of D) and budget restric- 
tion (defined in terms of C) do not restrict the evaluation, i.e. the target and 
the restriction are not met after two additional stages. The idea of the calcu- 
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lation is illustrated in Figure 2. The benchmark path (dashed line) is a line 
that goes through the point (D k _i, C^-i + Cs) and has slope 2/h(D k _i). The 
benchmark path corresponds to the decision completing the experiment after 
having only one additional stage. The candidate paths have two stages. After 
the first stage the expected location is {Dk-i + n k h(D k -i)/2, C k -\ + Cs + n k ). 
For the second stage the slope is 2/h(D k ). Because h(D k ) > h(D k -i), the 
candidate paths cut the benchmark path at point 

n _ n n k h(D k ^) , C s h(D k -i)h(D k ) 
D - D ^ + o + 2{h{Dk) _ h{Dk _ l)y (17) 

h(D k ) \ ^ 




h(D k ) - /i(£) fc _i) , 



The cut points are calculated for a large number of candidate paths. The candi- 
date path that has the earliest cut-point (the smallest D) with the benchmark 
path is chosen as the optimal one. The corresponding number of measurements 
n k is taken as optimal for D k _i when the stage cost is Cg. 



2.4 Approximately optimal sequential designs 



In order to find a general model for the optimal function of the cur- 

rent information D k _\ and the stage cost Cs, the optimal n k is calculated for a 
large number of (D k _i, Cs) pairs using the two-stage approximation described 
in Section 2.3. The current information D k _x has 104 unequally spaced values 
from 1 to 1000. The stage cost Cs has 35 unequally spaced values from 1 to 
1000. The approximately optimal n k is chosen among of 368 candidate val- 
ues 2, 4, 6, ... , 180000, 190000, 200000. The obtained data contain the approx- 
imately optimal n k for 3640 (D k _i,Cs) pairs. Figure 3 presents the contours 
of n k in the (D k _ 1: C 5 )-plane for the cloglog model. Figure 3 suggests that the 
contours are nearly linear in the log-scale when D k _i > 10. 

The following model is found to have an excellent fit with the data 

logn fc = a + (3\ogD k _i + 7logC 5 + 51ogD fc -i logCg. (18) 

The estimated model parameters a, (3, 7 and 5 for the logit, the probit and 
the cloglog model are presented in Table 2. Apart from the intercept term, 
the estimated models are quite similar. 

In the model (18) it is assumed that a — 1. When a is estimated, model (18) 
is written as 

logn fc = a + /3 log(aD fc _i) +7logCs + <Jlog(aD fc _i)logCs. (19) 
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candidate path 



cut point 



x <- 'benchmark path 




D k ,C k ) 



(D k _,,C k .,) 



D 

Fig. 2. Schematic illustration of comparison of design paths. The benchmark path 
marked by dashed line starting from (-Dfc-i, C^-i) corresponds to completing the 
experiment in one stage whereas the candidate path path (solid line) corresponding 
to having first stage of nt measurements and then completing the experiment in 
one stage. The candidate design has an additional cost of Cs compared with the 
benchmark design but because of the additional information obtained at the first 
stage, the increase in the information is faster at the second stage. The cut point of 
the paths is given by (17) and the -D-coordinate of the cut point serves as a criterion 
for comparing candidate paths with different n&. 



In practical experiments, the stage size n k calculated from (19) need to be 
rounded to the closest even integer. 
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Fig. 3. Optimal number of measurements for the next stage as a function of 
current information D^-i and stage cost Cs for the cloglog model. The dots indicate 
the 3640 {D^i,Cs) pairs for which the value of was determined. The contours 
are calculated from these numbers. 



3 Application to switching measurements 



The measurement data from (jKarvanen et all I2007I ) is reanalyzed in order to 



illustrate the efficiency of the presented approach. The data are available from 



the Royal Statistical Society Datasets Website http : //www.blackwellpublishing . com/rss/Volumes/ 
In the experiment, a sample consisting of aluminium-aluminium oxide-aluminium 
Josephson junction circuit in a dilution refrigerator at 20 millikelvin temper- 
ature was connected to computer controlled measurement electronics in order 
to apply the current pulses and record the resulting voltage pulses. The initial 
maximum likelihood estimates were fo und using a binary search algorithm 
(jKarvanen et alll2007l ; lKarvanenl . I2008I ). After that the experiment continued 
as sequential design where D-optimal covariates values were calculated from 



12 



Table 2 

Summary of estimated models (18). 

logit R 2 = 0.9992 





estimate 


standard error 


a 


1.01515 


0.00553 




1.43396 


0.00162 


7 


0.41042 


0.00134 


6 


-0.01388 


0.00039 



probit R 2 = 0.9990 

estimate standard error 
a 0.12859 0.00611 

P 1.41891 0.00179 

7 0.40324 0.00148 

5 -0.00949 0.00043 



cloglog 


R 2 


= 0.9934 




estimate 


standard error 


a 


0.48044 


0.01366 




1.34593 


0.0042 


7 


0.39711 


0.00331 


S 


-0.00778 


0.00102 



the current maximum likelihood estimates of the parameters a and b. The 
approximate cost-efficient solution was not available at the time when the ex- 
periment was carried out and the stage size was set according to an ad-hoc rule 
where the stage size was 100 at the beginning and was then increased by 10 
percent at each stage. The recorded data contains parameter estimates after 
each stage. In addition, the data contains the times needed for measuring and 
for computation. These times are plotted in Figure 4 as a function of stage size. 
It can be seen that there is a linear relationship with the stage size and the 
total time. This supports the usage of the cost model defined in Section 2.1. 
From the data it is estimate that the stage cost is 0.88167 seconds and the 
marginal cost of making one additional measurement is 0.00386 seconds. These 
numbers depend on the technical details of the measurement devices and can- 
not be generalized to other experiments. In order to apply the model (19) the 
stage cost need to be standardized C s = 0.88167/0.00386 = 228.4. 
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2000 4000 6000 8000 10000 

stage size 

Fig. 4. Time needed for measuring and for computing as function of stage size in 
the studied switching measurement. 

The design used in the experiment is compared to the cost-efficient design 
where the stage size is determined according to model (18). The initial esti- 
mation took 4.0 seconds and resulted estimates do = 0.380, b = —95.60 and 
D = 54.05. Now model (19) suggest the stage size 720 for the next stage 
while the stage size used in the experiment was only 134. Thus, the used ad- 
hoc rule leads to some losses in cost-efficiency and should not be used when it 
is important to minimize the time needed for the experiment. Further compar- 
isons are carried out by means of simulation. For the simulation, it is assumed 
that the final estimates a = 0.240 and b = —60.628 are the true parame- 
ter values. Starting from the initial estimates, the D-optimal covariate values 
and the cost-efficient stage size are calculated and new measurement data are 
generated. The simulation continues sequentially until D exceeds the final D 
achieved in the experiment. The resulting design path in (D, C)-plane calcu- 
lated as the median of 100 simulated paths are presented in Figure 5. It can 
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10000 20000 30000 40000 50000 



D 

Fig. 5. Comparison of the used sequential design and the cost-efficient sequential 
design in the (D, C)-plane. The circles show the stages of the design used in the 
actual switching measurement experiment. The squares show the path with the 
cost-efficient design calculated as the median of 100 simulated paths. The dots 
present the stages of the 100 simulated paths and give an idea on the variation 
between the simulation runs. It is instantly seen that the cost-efficient design saves 
time needed for the experiment. The dashed line shows the theoretical limit where 
the optimal covariate values are known from the beginning of the experiment. 



be seen that the cost-efficient design is close to theoretical design where the 
D-optimal covariate values are known from the beginning and only one stage 
is needed. It is also seen that the ad-hoc rule leads to updating the parameter 
estimates too frequently. The total time needed for the experiment was 497 
seconds, of which the cost-efficient design could have saved about 40 seconds. 
This is not a crucial saving in a single switching measurement experiment but 
has importance when a number of experiments are carried out. 
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4 Discussion 



Motivated by a practical problem from experimental physics, a framework 
for cost optimization in sequential designs has been formulated. Particularly, 
the approximately cost-efficient sequential designs for the logit, the probit 
and the cloglog model are determined. Cost-efficient designs for other link 
functions can be obtained in a similar way. The results can be directly applied 
to switching measurements and they may lead to substantial savings in the 
time needed for the experiment. In addition, the results are potentially useful 
in other experiments in physics, chemistry or engineering. Finding the specific 
applications will be a part of the future research. 



The proposed approach may be criticized for using several approximations: 
Information matrix J is compressed to its determinant D. The change in D 
is approximated by its expectation which itself is calculated numerically. The 
choice between design paths is based on a two-stage approximation. Finally, 
the optimal n& is obtained from the statistical model estimated from numer- 
ical data relying on all previous approximations. Naturally, the approxima- 
tions have been used because analytical results are difficult to obtain. Nev- 
ertheless, it seems that the analytical results would not significantly improve 
cost-efficiency compared to the approximate results. Simulations in Section 3 
suggest that the approximate optimal design performed well compared to the 
theoretical D-optimal design with known parameters. 



The idea of balancing by ISitter and Forbes! (119971 ) and ISitter and Wul (119991 ) 
could be worth trying at least in the early stages of a sequential experiment. 
The calculation of the balanced design complicates the approach and takes 
some computational time but might result in further savings in the total time. 



The concept of cost-efficiency narrows the gap between the theory of opti- 
mal design and the requirements of practical applications. When the cost is 
minimized, the most relevant criterion for the experiment is optimized. The 
presented cost model with a fixed stage cost can be easily extended to serve 
the needs of different applications. Due to the complexity of cost optimization, 
the determination of cost-efficient designs requires numerical approximations. 
The numerical calculations may be complicated, but as it was demonstrated, 
once the optimization is done, the results are easily applied to all experiments 
with similar cost structure. 
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